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April  1,  1978  through  March  31,  1979 

Contract  No:  N00014-75-C-0266 
Project  No;  NR-049-345 
Information  Systems  Program 
Office  of  Naval  Research 

Bennet  P.  Licntz,  Principal  Investigator 
Graduate  School  of  Management 
University  of  California,  Los  Angeles 

This  report  summarizes  the  work  during  the  period  April  1,  1978  through 

March  31,  1979  under  contract  N00014-75-C-0266.  The  persons  supported 

during  this  period  were  Professor  Llentz,  Professor  B.  Swanson,  several 

Ph.D  and  Masters  students  (Dr.Kweku  Ewusi  - Mensah,  Mike  Levy),  and 

secretarial  support. 


The  reports  produced  during  this  period  were  as  follows: 

1.  Llentz,  B.  P.  and  E.  B.  Swanson,  "Discovering  Issues  in  Software 
Maintenance,  Data  Management.  Vol.  10,  pp.  15-18,  1978. 

2.  Llentz,  B.  P.,  E B.  Swanson,  and  G.  L.  Tompkins,  "Characteristics 
of  Application  Software  Maintenance,"  Communications  of  the  ACM, 

Vol.  21,  pp.  466-471,  1978. 

3.  Llentz,  B.  P.  "A  Multistage  Model  of  Network  Usage,"  Isletme  Dergisi, 
Vol.  4,  pp.  193-203,  1978. 

4.  Llentz,  B.  P.  and  M.  Chen,  "A  Methodology  for  Long  Range  Information 
Services  Planning,"  Long  Range  Planning,  (to  appear  in  Summer  1979). 

5.  Llentz,  B.  P.  and  E.  B.  Swanson,  "On  the  Use  of  Productivity  Aids  in 
System  Development  and  Maintenance,"  submitted  to  International 
Conference  on  Software  Engineering,  1979. 

6.  Llentz,  B.  P.  and  E.  B.  Swanson  "A  Survey  of  Application  Software 
Maintenance."  Data  Management  (to  appear  in  1979). 

7.  Ewusi-Mensah,  W.  and  B.  P.  Llentz,  "On  the  Application  of  Peak  Load 
Pricing  to  Computer  Services,"  submitted  for  publication. 

8.  Llentz,  B.  P.  and  I.  R.  Weiss,  "Trade-Offs  of  Secure  Processing  in 
Centralized  versus  Distributed  Networks,"  Computer  Networks.  Vol.  2, 
pp.  35-43,  1978. 
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The  publications  highlight  the  major  areas  addressed  in  the  research  program: 
computer-communication  systems  and  software  maintenance.  In  the  computer- 
communication  area  the  work  addressed  applying  a simulation  nusdel  to  demon- 
strate the  ability  to  do  trade-off  analysis  between  security  measures  and 
computer  performance,  in  terms  of  throughput  and  response  time  as  well  as 
cost  Additional  work  was  done  in  the  area  of  peak  load  pricing  of 

network  services  The  emphasis  here  is  to  derive  models  for  develop- 

ing pricing  policies  using  economic  theory.  I»— (djT^^llfe  cycle  model  is 
presented  for  the  use  of  timesharing  and  interactive  systems. 


The  major  current  effort  is  in  the  application  software  maintenance  area. 
Data  collected  by  an  extensive  survey  has  been  analyzed  and  several  reports 
have  been  produced,  ^^ientz  and  Swanson  (1)  discuss  software  maintenance 


issues.  The  tabular  results  of  the  survey  appear  in  (6). 


One  particular  area  of  Interest  is  the  use  of  productivity  aids.  This  has 
been  documented  in  (5)  and  is  being  considered  for  presentation  at  the 
International  Conference  on  Software  Engineering.  Another  paper  that  is 
being  developed  reports  the  use  of  factor  analysis  in  determining  problem 
areas  that  affect  maintenance  effort.  Analysis  indicates  that  the  predom- 
inant group  of  factors  involve  user  needs,  expectations  and  participation. 

The  second  group  of  factors  Involves  management  issues  — resource  alloca- 
tion, etc.  The  third  group  consists  of  technical  issues,  documentation,  etc. 


The  pilot  survey  reported  in  (2)  consisted  on  the  first  phase  in  maintenance. 
The  second  phase  which  Included  the  expanded  survey  as  reported  in  (1),  (5), 
(6)  provides  an  overall  view  of  maintenance.  Current  effort  has  several 


foci: 


- Continuation  of  analysis  of  data 
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- appllcntlon  to  real  D«0.  Systems 

- research  stemming  from  survey  work 

The  first  of  these  has  been  addressed.  The  second  area  has  begun.  With 
the  help  of  ONR  staff  members,  contact  was  established  with  a maintenance 
group  of  the  U.  S.  Marine  Corps  at  Camp  Pendleton,  California.  An  initial 
visit  was  made.  Results  of  the  survey  were  presented.  It  is  intended 
that  this  will  be  followed  up  by  an  exchange  of  documents  and  then  by 
assistance  to  them  in  analyzing  their  current  maintenance  effort.  Addition- 
al visits  will  be  made  in  the  next  contract  period. 


Several  areas  of  research  have  emerged  from  the  survey  work.  One  area  is 
that  of  planning  to  control  maintenance  effort.  This  is  reported  in  (4). 

It  will  be  expanded  to  provide  a methodology  for  development  and  maintenance  i 

in  the  next  decade.  A second  area  is  that  of  creating  and  monitoring  3 

system  requirements  across  the  life  cycle.  This  is  being  worked  on.  The 

third  area  is  that  of  software  cost  estimation.  The  initial  estimation 

effort  will  be  in  the  maintenance  area.  It  will  consist  of  a classification 

method  of  systems  change  requests  and  measurement.  If  this  is  successful, 

an  attempt  will  be  made  to  extend  the  method  into  development. 

During  the  year,  a number  of  papers  were  presented.  Professor  Llentz 
presented  invited  papers  at  the  EDP  Auditors  Association,  the  ORSA/TIMS 
annual  meeting  and  the  Data  Processing  Management  Association  (DPMA). 

Professor  Swanson  presented  invited  papers  at  the  Systems  Science 
Symposium,  the  TIMS-ORSA  meeting,  and  the  DPMA. 
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The  remainder  of  this  report  sununarizes  some  of  the  recent  analysis  of  the 
maintenance  survey  data.  The  questionnaire  asked  managers  to  respond  to 
a variety  of  problem  factors  as  well  as  to  the  extent  of  maintenance, 
orm  problem  factors  accounted  for  88. 9X  of  the  problem  variance  using 
factor  analysis.  These  were 

“ programmer  effectiveness  (r“.191,  s=.001) 

- product  quality  (r=.191,  s=.001) 

- user  knowledge  (r=.ll3,  s=.006) 

- programmer  time  availability  (r«.101,  s=.013) 

Here  r is  the  correlation  measure  and  s is  the  significance  level.  Two 
other  factors,  system  reliability  and  machine  requirements,  were 
less  significant.  Each  of  these  factors  is  a cluster  of  specific 
problem  areas  identified  in  (6). 


The  problem  area  groupings  are: 
- user  knowledge 


- programmer  effec- 
tiveness 


- product  quality 


- programmer  time 

availability 

- machine  requirements 


- lack  of  user  understanding 

- inadequate  user  training  | 

- maintenance  programmer  productivity 

- maintenance  programmer  motivation 

- skills  of  maintenance  programmers 

- adequacy  of  system  design  specifications 

- quality  of  original  programming 

- documentation  quality 

- competing  demands  for  maintenance  time 

- program  storage  requirements 

- program  processing  time  requirements 
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Uslnp  analysis  of  variance,  the  six  problem  factors  were  considered 
against  the  level  of  staffing  sufficiency  and  the  demands  of  maintenance 
on  the  manager's  own  time.  It  was  found  that  four  problem  factors  varied 
significantly  by  perceived  level  of  staffing  sufficiency: 

- user  knowledge  (s=.007) 

- programmer  effectiveness  (s^.OOl) 

- product  quality  (s^.OOl) 

- programmer  time  uvailablllty  (a^*001) 

For  demands  on  the  manager's  own  time,  four  factors  varied  significantly: 

- user  knowledge  (s=.005) 

- programmer  effectiveness  (s^. 055) 

- product  quality  (s^.OOl) 

-system  reliability  (S-.076) 

These  results  indicate  that  system  reliability  and  machine  requirements 
have  a lesser  effect.  Several  potential  reasons  for  this  arc  improved 
price  performance  of  hardware,  migration  support  to  new  hardware  systems, 
and  age  indicating  system  stability. 

Analysis  was  performed  to  see  if  system  size  and  age  have  an  influence 
on  the  perceived  problems  of  maintenance.  It  was  shown  that  age  and 
size  Impact  the  magnitude  and  allocation  of  maintenance  effort  as 
expected.  Size  has  no  impact  except  for  only  machine  requirements  and 
system  reliability.  However,  these  accounted  for  only  11.1%  of  the  total 
problem  variance.  System  age  had  no  notable  influence  on  problem 


factors  whatsoever 


Analysis  of  variance  was  performed  for  the  six  problem  factors  and  system 
development  tools.  Of  these  factors  product  quality  was  affected  by  test 
data  generators,  structured  walk- through  and  chief  programming  team. 
However,  it  is  noteworthy  that  the  problems  of  user  knowledge  and 
programmer  effectiveness,  which  account  for  71.4%  of  the  total  problem 
variance,  arc  little  affected  through  the  use  of  these  techniques. 
Separate  analysis  revealed  that  greater  problems  of  system  reliability 
are  seen  to  be  associated  with  a data  base  dictionary.  This  may, 
perhaps,  reflect  the  complexity  of  the  system  software  incorporating  the 
tool.  No  significant  variances  were  found  when  the  six  problem  factors 
were  analyzed  with  the  use  of  a data  base  management  system. 

It  has  been  argued  in  the  literature  that  maintenance  effort  depends  on 
the  choice  of  language.  Using  one  way  analysis  of  variance  two  factors 
were  seen  to  vary  significantly  with  language.  These  were  programmer 
effectiveness  (s-.OlO)  and  system  reliability  (s^.OSO).  For  programmer 
effectiveness  problems  tend  to  be  greater  than  average  where  assembler 
languages  are  used  and  notably  greater  than  average  for  PL/1  and  FORTRAN. 
For  RPC  it  is  less  than  average.  COBOL  is  average.  This  is  not 
entirely  unexpected  since  sophistication  In  a language  may  affect 
programmer  effectiveness. 


